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Abstract 

Even as research documents that teachers matter, there is less certainty about the attributes of teachers 
that make the most difference in raising student achievement. Many studies have estimated the 
relationship between teachers' characteristics (i.e., experience and academic performance) and their 
value-added to student achievement. Few have explored whether instructional practices predict student 
test score gains. In this study, the authors ask what classroom practices, if any, differentiate teachers 
with high impact on student achievement in middle school English Language Arts from those with lower 
impact. The study further explores the extent to which value-added measures signal differences in 
instructional quality. Even with the small sample used in the analysis, the authors find evidence that 
high value-added teachers have a different profile of instructional practices than do low value-added 
teachers. Teachers in the top quartile as measured by value-added scores score higher than second- 
quartile teachers on all 16 elements of instruction that were measured. The differences are statistically 
significant for a subset of practices including explicit strategy instruction. 




Introduction 



Even as research has begun to document that teachers matter, there is less certainty about what 
attributes of teachers actually make the most difference in raising student achievement. Our own work 
and the work of others suggest that differences in teacher preparation may account for some of these 
differences, particularly in the first year of teaching, while other work identifies teacher attributes, 
including certification, the selectivity of teachers' undergraduate institutions, and teachers' scores on 
tests of general knowledge and verbal ability as factors that may be related to student achievement 
gains (Boyd et al. 2009; Kane, Rockoff, and Staiger 2005; see Rice 2003 for a review). The emphasis on 
teacher characteristics and preparation, however, obscures the importance of instruction within the 
classroom. Teachers' classroom practices are likely to be the mechanism by which teachers affect 
students. In addition, even if certain teacher attributes make a difference, such as having attended a 
more selective college, such findings may have limited utility for policy makers and teacher educators. 
Identifying classroom practices associated with high student achievement gains, and then targeting 
these practices in teacher education and professional development, provides a potential avenue for 
improving the quality of instruction for all students. 

While our first goal is to identify those elements of instruction that lead to improvement in 
student learning, we are equally interested in assessing whether value-added measures capture 
instructional differences among teachers. Measures of value-added to student test score gains are 
becoming more and more common tools for measuring teacher effectiveness (Sanders and Rivers 1996; 
Rivkin, Hanushek, and Kain 2005; Rockoff 2004). Yet, researchers have debated the validity of these 
measures, questioning whether value-added measures reflect the characteristics of the students in the 
classroom more than the contribution of teachers to student achievement (Rothstein 2007). This paper 
describes the pilot study of a larger effort to uncover the extent to which value-added measures reflect 
differences in instruction, as well to identify the classroom practices that are more characteristic of 
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more effective teachers. We focus on middle school English Language Arts instruction and test 
performance in New York City. To measure instruction, we used a structured observation protocol that 
combines new measures of instruction with measures that have been used elsewhere (LaParo, Pianta, 
and Stuhlman 2004; Pianta et al. 2006) in addition to collecting additional measures of instruction, 
including teacher logs and student work. In what follows we describe the differences in instruction 
between teachers with high value-added scores and those with moderately low value-added scores. 

In this study, we built on our earlier work on pathways into teaching and on our data base on 
New York City teachers to investigate the classroom practices associated with teachers whose students 
make above average gains in English/Language Arts, particularly in more challenging schools. We ask 
what classroom practices, if any, differentiate between teachers with high impact on student 
achievement in middle school English/Language Arts, as measured through value-added analyses, from 
teachers with lower low value-added scores? In addition, this study explores whether these kinds of 
value-added analyses represent a reasonable signal for differences in instructional quality and teacher 
effectiveness; are value-added measures, in fact, associated with observable differences in instruction? 

In order to investigate the classroom practices of middle school English/Language Arts teachers, 
we also need tools for classroom observation that are able to capture differences in instruction that 
might differentiate more effective teachers. In this study, we combined the use of several dimensions of 
the CLASS (Pianta, Hamre, Haynes, Mintz, and La Paro 2006), a widely used classroom observation 
system, with a new observation protocol we developed specifically for secondary English/Language Arts 
instruction. In this paper, we describe the origins of this instrument, its elements, and its use in a study 
of middle school ELA instruction in New York City. We also discuss the relationship between the 
practices targeted in our instrument and value-added assessments of teacher quality. Our goals include 
both identifying those elements of instruction associated with improvement in student learning and 
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investigating empirically the extent to which value-added measures capture differences in instructional 



quality among teachers. 



Framework for Instructional Interactions in English/ Language Arts 

Our study focuses specifically on instruction in English/Language Arts, as there is growing concern about 
adolescent literacy in the U.S. While students in elementary schools have shown gains in literacy, as 
measured by the National Assessment of Educational Progress, students at the 8 th and 12 th grade levels 
have demonstrated relatively few gains in reading. 1 Even more disturbing, the achievement gap 
between white and non-white students, and between higher and lower SES students, seems to be 
increasing (Snow and Biancarosa 2003). Though a number of curricular and instructional programs have 
been aimed at improving literacy (see Snow, Burns, and Griffin 1998 for an overview of 12 such 
programs), we still know relatively little about the instructional practices that support literacy 
achievement during middle school, particularly in urban schools. While we know that teachers in other 
content areas help develop students' academic literacy, English/Language Arts teachers are the teachers 
who are most explicitly charged with developing students' literacy skills. We focus on instruction in ELA 
classrooms where we can look at relationships between instruction and student achievement in reading 
and writing. 

Much of the existing research on middle schools has focused on the importance of school 
organization (e.g. blocked scheduling; interdisciplinary teams), opportunities for student choice 
(electives and exploratory classes) and on the creation of strong adult-student relationships through 
structures such as advisories (Carnegie Council on Adolescent Development 1989). While school-level 
structures can certainly impact student experiences and a variety of outcomes, we are most interested 



1 See report card here: http://nces.ed.gov/nationsreportcard 
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in classroom-level instructional interactions and how these interactions predict students' academic 
achievement. 

We focus on middle school because so little research has focused on instruction at this level. 

Yet we believe that middle school is a consequential time in students' academic lives (Carnegie Council 
1989) and that the quality of instruction students receive during this time affects their chances at 
success in high school and beyond. While there is a growing consensus on effective approaches to early 
literacy instruction (Snow, Burns, and Griffin 1998), there is much less agreement about effective 
literacy instruction for secondary school students, including middle grades. 

Prior research has suggested multiple dimensions that characterize classroom interaction 
including the intellectual challenge of tasks assigned to students (Newmann, Lopez, and Bryk 1998), the 
quality of instructional conversation, including teachers' uptake and elaboration of student ideas 
(Nystrand 1997; O'Connor and Michaels 1993); and representations of content, including the use of 
analogies or examples (Leinhardt 2004). In addition, measures of classroom practice must capture how 
those teaching English/Language Arts attend specifically to the needs of English learners in their 
classrooms as such students make up an increasing percentage of students in classrooms nationwide 
(Short and Fitzsimmons 2007). Such accommodations might include more scaffolding of tasks, support 
for academic language development, and targeted vocabulary instruction (Short and Fitzsimmons 2007). 
Measures of classroom practice must therefore be sensitive to a range of interactions around 
instruction, including the nature of the task, the focus of instruction, the features of classroom 
discourse, the types of accommodations provided, and the quality of feedback given to students. 

While some practices associated with higher student gains in English/Language Arts may be 
inherently subject-specific, such as comprehension instruction, other more generic factors may also 
contribute to teacher effectiveness at the middle school level. Student motivation and engagement, for 
example, have been strongly associated with student achievement in literacy, particularly at the 
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secondary level (Guthrie and Wigfield 2000). A long history of research on teaching also suggests that 



effective teachers may be better at capturing more time for academic instruction (Denham and 
Lieberman 1980) and keeping students focused on their tasks than less effective teachers. Effective 
teachers may have more efficient routines for transitions between activities, and better classroom 
management that result in more time for instruction. 

Given the multifaceted nature of effective teaching, our instrument takes many of these 
potential dimensions of instruction into account, while focusing on ELA specific instructional 
interactions. While some aspects of classroom practice are likely to be general, cutting across subject 
areas and grade levels, other features of instruction are almost certainly domain-specific, requiring 
subject-specific measures (Stodolsky 1988). One of the challenges of creating tools for measuring 
instruction that can be useful across a wide range of classrooms lies in addressing this balance between 
measuring both generic and more domain-specific elements of classroom interaction (Grossman and 
McDonald 2008). 

Review of Observation Protocols 

A number of current observation protocols have been designed to focus upon elements of classroom 
instruction that may be consistent across different grade levels and content area, examining a series of 
features that could be considered generic elements of teaching. For instance, Danielson's (1996/2007) 
Enhancing Professional Practice: A Framework for Teaching, focuses on teacher preparation and 
knowledge, standards based instruction, necessary material resources, and student and teacher 
relationships. Similarly, Pianta and his colleagues developed the Classroom Assessment Scoring System 
(CLASS) to assess instructional approaches, as well as the teacher-student, and student-to-student 
interactions and nature of the classroom environment (LaParo, Pianta, and Stuhlman 2004). CLASS 
focuses upon a number of key instructional dimensions that can be examined in different subject areas, 
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such as the degree to which a teacher helps the student understand ideas within a disciplines and the 
broader framework of the domain, and the degree to which a teacher provides students opportunities 
for analysis and higher order thinking skills, and the quality of feedback a teacher provides for students. 
The protocol also captures the emotional climate of the classroom and the teacher's attempts to 
address adolescent needs and perspectives. 

Other instruments have been designed to measure teachers' understanding of best practices 
and/or specific subject matter knowledge in content areas. Emphasizing the importance of making 
mathematics accessible to students, Hill and her colleagues developed the Mathematical Quality of 
Instruction instrument (MQI) to assess the accuracy and richness of teachers' mathematical ideas, 
language, representations and tasks (Hill et al. 2008). In literacy, a number of observation protocols have 
been developed around comprehension, including systems developed by Taylor et al. (2005) at the 
Center for the Improvement of Early Reading Achievement out of the University of Michigan. The TEX- 
IN3 system (Hoffman et al. 2004) focuses specifically on the presence and use of texts in classrooms. 

None of the existing observation protocols, however, provides a way to observe across the 
many domains of ELA classrooms, particularly at the secondary level. Both of the aforementioned 
literacy protocols focus specifically on one element of English Language Arts instruction— reading— and 
were designed for elementary classrooms. The paucity of discipline-based observation approaches has 
been a persistent problem in efforts to develop assessments of teaching (Kennedy, in press). Indeed, as 
Kennedy argues, "until recently, assessments have not attended to the intellectual substance of 
teaching: to the content actually presented, how that content is represented, and whether and how 
students are engaged with it. ..Documenting the intellectual meaning of teaching events remains the 
illusive final frontier in performance assessment" (p. 21). To that end, the PLATO instrument builds on 
existing observation tools and research on effective teaching practices in ELA in an attempt to parse the 
different facets of teaching practice in secondary ELA classrooms. 
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Development of PLATO 

The Protocol for Language Arts Teaching Observation (PLATO) is based on research on effective literacy 
instruction at the secondary level and is designed for observations of middle and high school 
English/Language Arts classrooms. 2 In its initial version, which we use for the study presented here, 
PLATO was designed to supplement our use of the CLASS instrument and therefore employed the same 
7-point scale referenced to 3 levels (low, medium, high). For each element, we developed indicators of 
interactions that would receive low (scores of 1 and 2), medium (scores of 3, 4, and 5), and high scores 
(6, 7) by raters (see attached instrument for full details). Researchers observed for 15-minute intervals, 
and then coded that 15-minute segment of instruction, according to both the PLATO elements and six 
CLASS elements described in more detail below. 

In its first iteration, PLATO included ten elements of effective English Language Arts instruction: 

• clarity of purpose of the lesson 

• level of intellectual challenge in both teacher questions and tasks assigned to students 

• representations of content 

• connections to both personal and prior knowledge 

• use of models and modeling of both high quality work and strategies for reading and writing 

• presence of explicit strategy instruction in reading and writing 

• use of guided practice in the classroom 

• quality of feedback offered to students by both teachers and peers 

• qualities of classroom discourse, including teachers' response and elaboration of student ideas 

• accommodations for English Learners 



2 See description of evaluation tool here: http://cset.stanford.edu/research/tools.html . 
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The element "Purpose" derives from research that suggests that children who learn in classrooms in 



which the purposes and goals of their work are clearly articulated, and the relationships between what 
they learn and broader goals are clear (Borko and Livington 1989; Smith and Feathers 1983). The 
element "Intellectual Challenge" was designed to focus upon the nature of the task and the degree to 
which it represents a developmentally appropriate stretch or reach for the specific students (Newmann, 
Lopez, and Bryk 1998; see also Vygotsky 1978). We also wanted to capture the ways in which teachers 
made content accessible to students and/or contextualized that content in terms of students' prior or 
personal knowledge. Based upon research that suggests that teachers' content knowledge— and the 
ways in which they represent content to children— may affect the ways in which students learn (Lee 
1995, 2007; Stevenson and Stigler 1994), we included the element "Representation of Content" in order 
to evaluate the teachers' disciplinary knowledge, and the accuracy of the representations of the content 
she or he made to students during the observed segment. When teachers connect new material to 
students' personal experiences and prior learning, students are more likely to develop a deeper 
understanding and make their own connections (Bransford and Johnson 1972; Tharp and Gallimore 
1988; Levin and Pressley 1981). Connections to personal and/or cultural experiences are particularly 
relevant for ELA instruction, in which students may be asked to make such connections to a literary text. 
To that end, the element "Connections to Prior/Personal Knowledge" evaluates the degree to which 
teachers make these linkages. 

Students also need examples of strong work in ELA, strategies to help them produce 
sophisticated readings and written texts, and structured and scaffolded opportunities to practice 
employing those strategies. Three additional elements, "Modeling," "Explicit Strategy Instruction," and 
"Guided Practice," attempt to capture this triad of practices central to effective ELA instruction. When a 
teacher provides examples and models of what students are being asked to do, students have specific, 
concrete images of what their work can and should look like (Frederickson and Collins 1989; Graham 
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2006; Hillocks 2000; Knudsen 1991). The element "Models/Modeling" captures whether there are 
models of quality work available in the classroom to guide student work and whether or not they are 
analyzed. It also assesses the extent to which a teacher names and then models specific meta-cognitive 
strategies or skills that she wants students to use, such as being able to figure out the meaning of a word 
using context clues, being able to write a strong opening paragraph for an essay, or how to edit 
someone else's writing. Because research also suggests teaching specific strategies that can be used 
flexibly across a range of ELA activities can enable students to be more successful, we included an 
element on "Explicit Strategy Instruction" (Beck and McKeown 2002; Greenleaf, Schoepenhauer, Cziko, 
and Mueller 2001; Palinscar and Brown 1987). The element of "Guided Practice," based upon research 
upon the role of practice with support (Vygotsky 1978; see also Lave and Wegner 1991), evaluates the 
level of support that a teacher provides in the segment observed as well as a teachers' capacity to check 
in with his or her students about their programs, evaluate their learning, and offer any needed support. 

The opportunities for extended discussion as well as the ways in which student ideas are 
responded to and clarified are equally important aspects of an ELA classroom. The "Feedback" element 
is based upon a long line of research that suggests that feedback facilitates student learning (Thorndike 
and Elliot 1931; Kluger and DeNisi 1996), particularly when it is specific and targeted (Sadler 1989; 
Sperling and Freedman 2001). This element captures when teachers elicit student ideas, probe student 
thinking, and have opportunities to address misconceptions. The "Classroom Discourse" element grows 
out of research on the nature of productive discourse that can promote learning in the classroom 
(Nystrand 1997; Nystrand and Gamoran 1991; O'Connor and Michaels 1993; Taylor, Pearson, Peterson, 
and Rodriguez 2003) as well as upon research on the typical discourse pattern of "initiation-response- 
evaluation", and a tendency to pose lower-order, rote or recall questions that do not require analytic 
thinking (Cazden 2001; Mehan 1979). Thus the "Classroom Discourse" element assesses opportunities 
students have for conversations with the teacher or among peers, as well as whether the discourse is 
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perfunctory and minimal at the low end, or elaborated, and purposeful at the high end. Building upon 



work that suggests that introducing academic language in classrooms can help bridge students' home 
discourse with the language used in school (Delpit 1988; Schleppegrell 2004), this element also focuses 
upon whether or not the teacher introduces specific ELA language and concepts and the degree to 
which he or she supports students in using those terms. 

Finally, building upon work that suggests that teachers in most mainstreamed classrooms today 
are increasingly responsible for teaching English learners, and hence, must be able to respond both to 
their language needs as well as support their academic development (August and Hakuta 1997), we 
developed the dimension of "Accommodations for language learning" in order to capture the range of 
strategies and supports that a teacher might use to make a lesson accessible to non-native speakers. 
Accommodations we wanted to capture included teachers taking into account individuals' levels of 
language proficiency, strategic use of primary language, grouping strategies, differentiated materials 
and assessments, as well as graphic organizers and visual displays. 

Because content coverage can also be an important predictor of student achievement (Rowan, 
Correnti, and Miller 2002), we wanted to capture different aspects of teachers' curricular focus during 
instruction. For this reason, PLATO also includes checklists for the major content domains within 
English/Language Arts including reading, writing, literature, speaking/listening, and grammar and 
mechanics. For each segment of instruction, observers check the domain that was the focus of 
instruction, and identify additional features related to that domain. For example, if observers identify 
reading as the target domain, they also identify the nature of the text read (e.g. fiction or non-fiction), 
the focus of reading instruction (e.g. comprehension, decoding, metacognitive strategies, etc.), and the 
nature of in-class reading activity (e.g. independent reading, teacher reading aloud, etc.). 
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Research Design 

Sampling Procedures 

In order to select teachers for this pilot study, we began by estimating the value-added to student 
achievement of all New York City teachers teaching sixth, seventh or eighth grade classes that take the 
English Language Arts exam. There are active debates concerning the best specification for estimating 
teacher effects. Because there is no consensus on the best approach, we chose to combine two 
measures. In particular, we used one estimate that includes student effects and models gains in student 
achievement as a function of student fixed-effects, student time-varying characteristics, (such as 
whether a student changes schools), school characteristics, classroom characteristics, and year and 
grade indicator variables. This strategy identifies value-added by comparing teachers who teach the 
same students, usually in different years. Our other estimate includes student controls, school controls, 
classroom controls and year and grade indicator variables. The student controls include gender, race, 
eligibility for free lunch, prior year test scores in math and ELA, and English learner status, among other 
factors. Classroom variables include the aggregates of all the individual variables plus the standard 
deviations of the prior year test scores. The school variables include enrollment, the percent of both 
black and Hispanic students, the percent of English learners, and the school average expenditures per 
pupil. We shrink each measure of value-added using empirical Bayes techniques to adjust for estimation 
error in calculating value-added. 

We then divided teachers into quartiles based on each of these two estimated value-added 
measures. In particular, we identified teachers in their third through fifth year of teaching who were in 
the second quartile of value-added performance on both measures or were in the fourth quartile (the 
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top) of value-added performance on both measures. 3 Using these samples, we identified matched pairs 
of middle-school teachers - one moderate-performing (2 nd quartile) and one high-performing (4th 
quartile) - teaching in the same school. We choose 12 pairs, 24 teachers, for the pilot. Neither observers 
nor participants knew the value added quartiles of specific teachers during any component of data 
collection. 

We focused on third through fifth year teachers because we have been following the cohort of 
teachers who were in their fourth year at the time of this study since they entered teaching in 2004; 
from our earlier work, we know a significant amount about their preparation, entry into teaching, 
perceptions of school context, and early experiences in the classroom. In addition, research on teaching 
also suggests that by the fourth or fifth year, teachers have developed a more stable set of instructional 
practices, and many teachers have reached a stage where they begin to plateau in their impact on 
student achievement (e.g. Boyd et. al. 2006; Kane, Rockoff, and Staiger 2005). However, there were not 
enough teachers from this single cohort that met our stringent requirements (value-added and matched 
in schools), so we included one cohort with an additional year of teaching experience and one cohort 
with one year less of teaching experience. 

Table 1 describes the teachers in our sample. 83 percent of the teachers are female. 
Approximately 21 percent of the teachers are black, four percent are Hispanic and 54 percent are white. 
Half of the teachers entered teaching through traditional teacher education programs (college 
recommending) and a quarter through the largest alternative route in NYC, the NYC Teaching Fellows. 
The remainder of the teachers entered through a variety of pathways including Individual Evaluation, 
Teach for America, and the Temporary license process. 



3 We selected teachers in the second quartile rather than the lowest because we thought that there might be 
relatively little instruction occurring in the lowest-quartile classrooms and hence less to be learned. In addition, 
the differences between teachers in the top and bottom quartiles might have been so striking that they might have 
alerted raters to a teacher's quartile during observations, potentially biasing their scoring of classroom. 
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Data Sources 



The primary data source for this study is structured classroom observation. In addition to PLATO, 
described above, we used 6 elements from two domains the Classroom Assessment Scoring System 
(CLASS) (La Para, Pianta and Stuhlman 2004) to assess two of the more generic aspects of instruction we 
wanted to measure- emotional support and classroom organization. Within the domain of Emotional 
Support, we measured Positive Climate, Negative Climate, and Regard for Adolescent Perspectives. 
Within the domain of Classroom Organization, we included Behavior Management and Productivity. We 
also included the CLASS measure of Student Engagement. All of our observers were trained and 
certified in the use of CLASS prior to entering the classrooms. 

We assessed inter-rater reliability on PLATO consistent with the procedures used for CLASS. We 
first identified target scores for each video, based on master coders, and then calculated the percentage 
of ratings by individuals that fell within 1 point of the target score. All observers completed training both 
in CLASS and in PLATO. Reliability for using PLATO was assessed using at least 5 different videos of 
classroom instruction across three different content domains (writing, literature, and grammar). All 
observers achieved at least 80% reliability on both CLASS and PLATO before observing in the field. We 
re-assessed inter-rater reliability during the second wave of observation and again achieved a minimum 
of 80% agreement on our ratings. 

We observed teachers on six separate days during the spring of 2007. On each day, we 
observed teachers for at least two hours of instruction, generally in two different classes. The six days 
were divided into two waves of data collection, separated by between two to six weeks. As mentioned 
above, observers did not know the quartile of the teachers they were observing. 

We worried, however, that using structured observation instruments alone might cause us to 
miss other aspects of classroom practice not included on the protocols that may in fact distinguish the 
highly effective teachers. To capture additional dimensions of classroom practice, including 
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relationships among teachers and students, peer interactions in the classroom, curricular focus, 
evidence of links to out-of-school literacy practices we also included more open-ended observations. 

For this reason, we had two observers in the majority of classrooms; one observer in the classroom took 
open-ended notes while the second observer used PLATO and CLASS. 

To gain insights over an extended period, we supplemented six days of direct classroom 
observations with teacher logs based, in large part, on the Study of Instructional Improvement's teacher 
log for ELA for middle school classrooms (Rowan, Camburn, and Correnti 2004). The log replicates the 
content categories of the PLATO, allowing us to coordinate two of our measures of instruction. We 
asked teachers to fill this log out for 15 consecutive days of instruction, or roughly three weeks. 

Teachers began to fill out the logs on the days we began our observations, which allowed them to ask 
any questions once they began to use the log. With these logs, we are also able to assess the overlap 
between observer's and teacher's assessment of content coverage and activity structures for the days of 
observation. 

In addition to the log, we also collected both assignments and student work samples to get 
another vantage point on instructional practice. We adapted the protocols for collecting and analyzing 
student work developed by Newmann and his colleagues (see Table 2). Because prior research suggests 
more challenging assignments are positively correlated with higher student performance (Newmann, 
Lopez, and Bryk 1998), we collected copies of assignments teachers designated as challenging and 
typical for their classroom. For each assignment, we collected the work from students that teachers 
considered high, average, and low performing (2 students from each group). For each teacher, we had 
12 pieces of student work. We then coded qualities of the both the assignment and of the written work, 
using protocols adapted from Newmann and his colleagues. 
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Findings 

The distribution of scores across observations varies for both the PLATO and the CLASS elements (See 
Figures 1 and 2). 4 On average, teachers received lower scores for PLATO elements across the board 
than on CLASS elements (see Tables 3 and 4). This result confirms other studies using CLASS alone, in 
which teachers tend to score higher on the domains of Emotional Climate and Classroom Organization 
than they do on the CLASS domain of Instructional Support. Of the PLATO measures, on average, the 
teachers scored highest on Purpose and lowest on ELL Accommodation. The standard deviation across 
teachers is approximately 1.0 for all ten PLATO elements. For the CLASS elements, the teachers in our 
sample scored highest on Behavior Management and lowest on Negative Climate, which is the one 
element constructed to represent a negative classroom behavior and is reverse coded. The CLASS 
elements have somewhat higher variances across teachers than the PLATO elements. 

Our structured observation data suggests systematic differences between teachers in the two 
value-added quartiles. Figure 3 illustrates the average PLATO scores for teachers by their value-added 
group. Across all PLATO elements, the high value-added teachers (top quartile) scored higher on average 
than the low value-added teachers (2nd quartile). This difference is evident even though observers 
were unaware of the value-added quartile of their subjects when scoring instructional practice. Figure 4 
illustrates the average scores for teachers for CLASS dimensions by their value-added group. The small 
size of the sample (24 teachers) makes it difficult to statistically differentiate groups; however, T-tests of 
the differences across groups on each of the PLATO scores shows that the groups are statistically 



4 Of the PLATO elements, Purpose, Intellectual Challenge and Representations of Content have approximately bell- 
shaped distributions. Connections to Prior/Personal Knowledge has an approximately uniform distribution, except 
that few observations achieved the very top score. The remaining measures are positively skewed across the 
observations. Among the CLASS measures, Positive Climate, Adolescent Perspectives, Productivity, Student 
Engagement are approximately bell-shaped, while Negative Climate is positively skewed and Behavior 
Management is negatively skewed. See Figures 1 and 2 for distributions. 
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different on the element of Explicit Strategy Instruction (p=0.03) and close to statistically different on 



the elements of Guided Practice (p=0.09) and Intellectual Challenge (p=0.13). 

The high value-added teachers also received more positive scores on each of the six CLASS 
elements, with the exception of Negative Climate, which is lower among this group (see Figure 4). 
However, only the difference in Student Engagement across the groups is significant at the .05 level, 
although Negative Climate is close (p= 0.13). 

Explicit Strategy Instruction 

One of the most striking findings was the association between a teacher's score on the Explicit Strategy 
Instruction element and his/her value-added quartile. However, many elements of PLATO are positively 
correlated with each other (see Tables 5 and 6), potentially confounding our understanding of the 
relationship between instructional elements and value added quartiles. In order to provide a somewhat 
clearer picture of these relationships, Table 4 provides the results of simple logits predicting value- 
added group as a function of PLATO scores on multiple dimensions. This set of analyses, in particular, 
assesses the extent to which the relationship between Explicit Strategy Instruction and teachers' value- 
added scores holds up to the inclusion of other measures of instruction. Each column within each of the 
two vertical panels of the table represents a separate model in which Explicit Strategy Instruction and 
one other observational measure of instruction predicts the log odds of a teacher being in the high 
value-added group. As an example, in the first specification, a teacher with a one unit higher score in 
Explicit Strategy instruction is 4.88 times more likely to be in the high value-added group. This positive 
relationship between value-added and Explicit Strategy Instruction holds up to the inclusion of every 
one of the other measures of instruction. None of the other measures have nearly as strong a 
relationship with value-added, once Explicit Strategy Instruction is included in the model. 
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As is clear from this analysis, Explicit Strategy Instruction, in particular, appears to distinguish 
the more effective teachers in our sample. To get a better sense of what such instruction looks like, we 
provide several examples from the field notes taken during open-ended observations. Teachers who 
scored "high" (a score of 6 or 7) on Explicit Strategy Instruction provided students with very structured 
and specific ways to approach ELA activities. For example, one high quartile teacher systematically broke 
down a newspaper article on "skinny jeans" to help students understand the features of effective 
journalism. She instructed them on how to compose a list of "4 Ws" (who, when, where, and what), how 
to use that list to create a focused lead, and then how to incorporate supporting details culled from 
graphic organizers. Students then wrote their own newspaper articles with an arsenal of specific 
strategies. This focus on how students could tackle ELA tasks was reflected in other high quartile 
teachers' classrooms. One teacher carefully broke down the steps involved in using context clues to 
determine the meaning of unknown words (reading the sentence before and after the sentence at hand, 
substituting in multiple words etc.). Another taught students how to identify the passive voice in their 
own writing and then change it to the active voice. Thus these teachers made visible the often invisible 
process requisite for successful, sophisticated literary analysis, reading comprehension, or writing. 

Content Domain Findings 

In addition to collecting information on instructional practice, the observers recorded the content 
domain(s) covered in the lesson. Lesson segments could be coded for multiple content domains. On 
average, 31.2 percent of the observations focused on reading, compared with 43.3 percent on writing, 
38.5 percent on literature and 19.8 percent on speaking and listening. During reading instruction, we 
observed that the majority of the reading texts were fiction, the primary instructional focus was on 
comprehension rather than decoding or evaluation, and in about 23 percent of the observations reading 
was done in class either independently or in small or whole class groups. 
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We also noted differences in content coverage based on a teacher's value-added quartile. 



Within our sample, high quartile teachers are more likely to teach across content domains and are more 
likely to focus on writing and speaking, while low value-added teachers are more likely to focus their 
instruction on reading and literature. On a more detailed note, the low value-added teachers are more 
likely to use class time for individual reading and reading aloud, while the high value-added teachers are 
more likely spend time on pre-writing and having the students make oral presentations. None of these 
differences are statistically significant at the .05 level, which is perhaps not surprising for such a small 
sample. 

One possibility is that the differences that we see in instruction between high and low value- 
added teachers (as shown above in Figures 3 and 4) is driven by differences in the content domains. 
Perhaps, for example, all teachers teach better when they teach writing than reading and the higher 
scores on PLATO and CLASS are simply driven by high value-added teachers teaching more writing. In 
fact, this is not the case. If anything, instructional quality appears lower in writing than in other 
observations. Across almost all of the 16 elements, the writing observations have less positive scores. 
The differences are statistically significant for Representations of Content, Connections to Personal/ 

Prior Knowledge, Feedback, Positive Climate, Negative Climate, Adolescent Perspectives, Behavior 
Management, and Productivity. Only in the element of Modeling, does writing instruction appear better 
than instruction in other content areas. 

In contrast to lower scores during writing instruction, teachers in both quartiles scored 
significantly higher on the majority of elements during literature instruction. These differences are 
significant across the board except for modeling, explicit strategy instruction, and positive and negative 
climate. There were few differences in instructional quality as measured by PLATO and CLASS between 
observed segments of teaching reading and teaching other content domains. 
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Looking Across Measures: Examination of Teacher Logs and Student Work 

In addition to analyzing differences in instruction, as measured by PLATO and CLASS elements, we also 

examined teacher logs and student work. 

Teacher Logs: The teacher logs indicate some differences between teachers in the two quartiles 
in content coverage and grouping strategies. Mirroring the observational results, high value-added 
teachers in our sample focused less on reading and more on writing and research skills than low value- 
added teachers. While only the reading and research differences are even close to statistical significant 
at usual levels (see Figure 5), the sample is small and the similarity in results between the logs and the 
observations suggest a trend worth further exploration. The logs also indicate differences between how 
teachers in the two quartiles use grouping structures (see Figure 6). Though teachers in both quartiles 
use independent work time for approximately the same percentage of instructional time, there were 
statistically significant differences between the frequencies with which teachers in the two quartiles 
used small group versus whole class instruction. In particular, high value-added teachers use small 
groups far more than low value-added teachers (36 percent compared with 16 percent), and they use 
large groups far less (26 percent compared with 44 percent. 

Student Work: Finally, we draw on student work. We coded the work for feedback, 
construction of knowledge, extended writing, disciplined inquiry, value beyond school, representations 
of content, and scaffolding. Across all measures but one, high value-added teachers score higher than 
low-value added teachers, although none of the differences are statistically significant. 

Beyond determining the instructional features that predicted teacher's value-added quartile, we 
are interested in the relationships between the different measures of instruction. Do different 
measures, such as structured observations, self-reported teacher logs, and student work assignments 
and samples pick up similar or different aspects of instruction? We found several interesting 
connections between the PLATO/CLASS observations and student work data. For example, teachers who 
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provide models or rubrics to illustrate 'good' work and define what constitutes quality in a specific 
lesson or domain are more likely to also give students feedback on how to improve their (r= 0.44, p< 
.05). In a way, modeling sets the stage for the feedback teachers provide when the assignment is 
complete. If students have greater clarity on a teacher's expectations for their work, the teacher can tie 
his/her feedback to the rubric or model he/she has provided and is thus better to able to give 
constructive, targeted feedback on that work. 

Modeling is also positively correlated with students' opportunity to construct knowledge, 
whether or not the assignment requires basic recall of facts or asks students to provide rationales for 
their analyses and interpretations (r= 0.44, p< .05). This suggests that teachers who provide a clearer 
sense of their expectations may be better able to push students for higher levels of analysis and 
interpretation perhaps because the assignment/instruction is better structured and goal-directed. There 
is also the possibility, however, that teachers who provide assignments that only ask for basic recall of 
facts do not see the need for models or rubrics because they are looking for right or wrong answers 
rather than more elaborated written work that could be stronger or weaker on several different 
dimensions. 

Interestingly, scaffolding for student work assignments is significantly correlated at the .1 level 
with several PLATO dimensions including feedback (r=.39, pc.l), classroom discourse (r=.37, p< .1), and 
modeling (r= .37, p< .1). Scaffolding looks at whether or not the assignment provides "structural 
supports, such as graphic organizers, multiple drafts, etc., that support further investigation and 
revision." It seems logical that teachers who are clearer about their expectations for an assignment, 
indicated by a higher score on modeling, would be better able to provide appropriate scaffolds to 
complete that assignment. In the same way, teachers who engage in "frequent back and forth 
exchanges with students about their work" and provide "specific and timely feedback that 
acknowledges what students did well and problems/incomplete understanding" (PLATO rubric) are the 
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same teachers who scaffold the completion of assignments, as multiple drafts provide multiple 
opportunities for teacher feedback. 

While our analyses of the teacher logs and student work samples demonstrate some 
connections across measures, they also highlight important differences among the measures. Teachers' 
self-reports of content coverage in the teacher logs differ significantly from our structured observations, 
indicating teachers perceive what they are doing differently from outside observers. For example, on 
days in which we observed, we noted one teacher teaching grammar over 30% of the time, and she 
reported teaching it only 7% of the time. While there are some interesting links between PLATO 
elements and features of the assignments teachers provide for students, as noted above, we observe 
surprisingly few connections between the features and/or quality of student work samples and PLATO 
or CLASS elements. This may be a result of the fact that the work samples represent the end product of 
instruction, while classroom observations represent the process leading up to the product. In this way, 
the two measures provide very different lenses on instruction. Moreover different measures collapse or 
disaggregate different instructional elements making comparisons across measures potentially more 
challenging. 

Discussion 

PLATO and Value-Added Measures 

This paper describes a pilot study to assess the relationship between classroom instruction and 
measures of teachers' value-added to student achievement based on students' test score performance. 
Even with the small sample used in our analysis, we find consistent evidence that high value-added 
teachers have a different profile of instructional practices than low value-added teachers. Teachers in 
the fourth (top) quartile according to value-added scores score higher than second quartile teachers on 
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all 16 elements of instruction that we observed, and these differences are close to statistically significant 



in certain elements such as intellectual challenge and guided practice. 

Our open-ended observation notes provide vivid illustrations of the range in the intellectual 
rigor of the instruction teachers provide. Instructional segments that score "low" (scores of 1 or 2) on 
intellectual involve students writing instructions for what they do when they wake up in the morning or 
completing highly formulaic "bio poems" that required little more than filling in the blanks. In sharp 
contrast, instruction that scored high on Intellectual Challenge (scores of 6 or 7) included students 
writing five paragraph essays about My Antonia, generating alternative endings to short stories, or 
crafting speeches from the perspective of presidential candidates. Instruction that scores low on Guided 
Practice either does not provide opportunities for students to practice new skills in class or allotted time 
for students to work independently but without sufficient support or "guidance" during class time. 
Lesson segments that score high on Guided Practice involve teachers circulating during literature circles 
answering student questions and clarifying their ideas or doing periodic whole class "check ins" as 
students work through stages of the writing process. In these elements, we find a clear relationship 
between a teacher's value-added quartile and several facets of high quality instruction. 

Why is Strategy Instruction Essential? What Does it Look Like in Practice? 

The dimension of explicit strategy instruction strongly differentiates teachers at the different levels of 
value-added and confirms research in the area of reading comprehension on the importance of strategy 
instruction (Pearson and Fielding 1991). In order to be successful on standardized assessments, the 
primary tool for determining a teacher's value-added score, students must consistently employ 
strategies for interpreting literary text, making a compelling argument, or analyzing grammatical errors. 
When students understand when and how to use specific strategies, as well as why they are useful, they 
may be better able to use such strategies on less familiar tasks or material. In contrast, students who 
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write editorials without a broader understanding of how to craft persuasive prose or build an effective 



paragraph may encounter more difficulties on the open-ended writing questions included on the 
majority of standardized assessments. 

Unfortunately, instances of effective strategy instruction are rare, as the positively skewed 
distribution of scores indicates. The mean score for Explicit Strategy Instruction, 2.1, is the lowest of all 
the PLATO elements except ELL accommodations, and the modal score is a 1. Even instructional 
segments that score high on other elements such as Intellectual Challenge, score lower on Explicit 
Strategy Instruction. During a lesson in which students were asked to "to anticipate an opponent's 
counter argument in writing an editorial," undoubtedly an intellectually demanding activity, the teacher 
did not discuss how students might accomplish the lesson's goal when writing independently. In fact it is 
during intellectually rigorous activities just beyond students' intellectual "comfort zone" that teachers 
most need to equip students with specific strategies. However, the vast majority of teachers provided 
students with directions for completing activities, but they did not instruct them on the nuances of how 
to complete those activities effectively. In literature circles, students were often told to analyze a 
character's actions or determine the meaning of unknown words without any discussion of the 
strategies that would enable them to do so. Similarly, teachers highlighted the features of cinquains or 
editorials but did not teach students how they might approach different types of writing based on those 
features. Thus the goal of many lessons was completion of the specific task rather than mastering a 
more broadly applicable skill or strategy. 

What Makes Writing Instruction More Problematic and Literature Less So? 

High value-added teachers were also more likely to teach writing than their less effective colleagues. 

Yet, writing instruction seems to provide another challenge for teachers. Across quartiles, teachers' 
scores on both PLATO and CLASS elements went down during writing lessons. Interestingly the only 
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PLATO element on which teachers on average scored higher during writing instruction was modeling. 



Writing provides the opportunity to generate a concrete model or exemplar (student work, published 
pieces, or teacher's own writing). Moreover providing a model for a writing strategy (brainstorming, 
organizing etc.) may be more familiar for many middle school teachers than modeling reading strategies 
that involve more metacognition (what a teacher is thinking when he/she is reading a text or 
interpreting literature). 

The lack of strategy instruction discussed previously was particularly pronounced during writing 
instruction. Our qualitative data indicates a clear distinction between "writing instruction" and lessons 
during which students were asked to write. Unfortunately we saw few of the former, regardless of 
teacher quartile. The majority of what were coded as writing lessons were lessons in which students 
spent class time writing but were given little to no direction about how to structure their writing or 
strategies to improve their writing. For example, at the conclusion to an introductory lesson on the 
features of poetry that highlighted a number of academic terms including simile, onomatopeia, and 
stanza, a teacher instructed students to "do a pre-write in poetic form." After students asked number of 
questions about the specifics of the assignment, she instructed them to, "write whatever [they] want 
about poetry- turn to a clean page in [their] journal and title it, 'Free Write: Poetry.'" A significant 
number of the writing assignments consisted of little more than "do nows" written on the board at the 
start of class. 

This lack of structure was evident in lessons across the stages of the writing process. The 
majority of peer editing sessions involved students reading each other's writing without specific features 
on which they should focus or questions to use to guide the editing process. As a result, students often 
provided each other with general or vague feedback such as "I love it!" or "you could make this better." 
Often teachers told students to "work on" or revise a draft for an entire class period with no 
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specifications or guidance about how to structure their efforts. As a result, we saw numerous students 



simply typing or copying earlier drafts in neater handwriting in an attempt to revise. 

Not only were the PLATO dimensions of practice lower in writing lessons; all of the social- 
emotional CLASS elements scored significantly lower during writing lessons, other than Negative 
Climate, which was significantly higher. As noted, the observed writing lessons tended to be less 
structured, with students spending a great deal of time working independently. This lack of structure 
might make writing lessons more difficult to manage and keep productive. When students were 
supposed to be writing independently, we noted the teacher's focus was heavily on behavior 
management. Regardless of the effectiveness of these efforts, highly visible behavior management, such 
as circulating with a notebook grading students on behavior, seemed to demand the vast majority of the 
teacher's time during writing lessons. We saw few instances in which teachers meaningfully 
conferenced with students about their writing; management seemed to take precedence. 

In sharp contrast, teachers' scores were significantly higher scores on seven of the ten PLATO 
elements during literature instruction. Literature lessons tended to be more structured around 
comprehension questions or reading texts aloud and analyzing as they went. This inherent structure 
may have facilitated teachers' ability to engage in higher-level discourse with students, make 
connections to students' experiences and prior knowledge, and demonstrate their own content 
knowledge through the use of examples and/or analogies. In addition, literature lessons involved more 
whole class and small group instruction, which may have made them easier to manage and keep on task, 
while writing lessons often had students working independently. 

Perhaps the general instructional challenges during writing instruction result from the fact that 
most secondary ELA teachers have degrees in English literature, and are thus more confident and 
competent with content related to literature rather than writing. English majors may be more familiar 
and hence more comfortable discussing theme or character in a novel than explaining the intricacies of 
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persuasive rhetoric and teaching students how employ those techniques in their own work. Yet, New 



York City has invested considerable resources around the teaching of writing in recent years, so these 
findings regarding the challenges of writing instruction remain puzzling. 

Refining the Instrument and Future Research 

The improvement of classroom instruction is at the heart of improving outcomes for all students. Yet, 
researchers and professional educators have been hampered by the lack of common tools for measuring 
classroom practice. In a recent article on assessment of teacher quality in practice, Ball and Hill (2009) 
commented: 

"The current enthusiasm for teacher quality requires caution. In the end, what matters is the 
quality of the instruction that students receive- that is, teaching quality. . .However, given the 
underdevelopment of the field right now, we need to improve the precision with which we 
conceptualize and measure teacher quality. . . We will have to delve into instruction and then 
map backward and forward to specific elements that we can use to predict instructional practice 
and its quality." 

Our efforts to develop a tool for classroom observations for secondary English Language Arts represent 
a beginning contribution to this effort. 

Ultimately, we hope to create a tool that is not only useful for research on teaching, but can be 
used for teacher development as well. By identifying components of classroom practice that are related 
to student achievement, we hope to contribute to the preparation of future English teachers and the 
ongoing development of practicing teachers. This tool might also prove useful in clinical supervision of 
English teachers, as it provides a common language for discussing and analyzing classroom practice. In 
a field that still lacks a technical language to describe practice (Lortie 1975), tools that help coordinate 
how we view classrooms and calibrate our sense of qualities of practice are sorely needed. 
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The tool is still far from perfect. For example, in our effort to reach across the domains of ELA, we ended 



up with more generic elements than we might have originally imagined. In addition, while each of the 
16 elements used in this study appears to signal a higher value-added teacher, they do not necessarily 
reflect different features of instruction. The elements are highly correlated, particularly at the teacher 
level. Because of these high correlations, we cannot necessarily conclude that it is one or more 
particular elements that drive the relationship between the observations of instruction and value added. 

Based on this pilot study, we have significantly revised this instrument, developed broader 
instructional factors within PLATO, collapsing, disaggregating, and eliminating elements that are highly 
correlated with one another, and changed the scoring from a 7 point to a 4 point scale. We recently 
completed using this revised version of PLATO in a follow-up study of 177 teachers in New York City 
middle schools to see if these initial findings hold up. 

Overall, our study provides evidence that value-added measures do more than measure the 
characteristics of students that arrive in a teachers' classroom. They also seem to be capturing 
important differences in the quality of instruction. This said, the small sample size of this pilot study 
limits our ability to identify with precision the instructional practices that most directly impact student 
achievement gains. Nonetheless, our findings regarding the impact of Explicit Strategy Instruction 
suggests that future research can begin to develop measures of teaching that can identify the attributes 
of instruction that can make a difference to student achievement. It is also possible that the impact 
some aspects of instruction, such as the quality of classroom discourse that may be important in 
developing students reasoning abilities and conceptual understanding of literature and writing, may not 
be measured well by the tests used to construct value-added scores. In addition to developing multiple 
measures of instruction, we need to develop multiple measures of student outcomes to ensure that 
classroom instruction supports the development of a broad range of learning outcomes for students. 
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Figures and Tables 



Table 1. Teachers in the Sample 



Variable 


Mean 


Female 


0.833 


Race/Ethnicity - Black 


0.208 


Race/Ethnicity - Hispanic 


0.041 


Race/Ethnicity - White 


0.541 


Year of Birth 


1975.1 (5.5) 


General Knowledge Exam 


254.6 (24.2) 


Pathway -College Recommended 


0.542 


Pathway- Individual Evaluation 


0.042 


Pathway - NYC Teaching Fellows 


0.250 


Pathway -Teach For America 


0.042 


Pathway - Temporary 


0.042 


SAT Math (n=13) 


458(70.7) 


SAT Verbal (n=13) 


501(77.5) 
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Table 2. Measures of assignments from Consortium for Chicago School Reform's Manual of Scoring Tasks and Student Work in Writing, 1998. 




Construction of 
Knowledge 



Assignment asks for simple 
recall of basic facts. 



Assignment asks for 
students to recall facts and 
give their interpretation. 



Assignment asks students to 
interpret or analyze and push for 
rationales or an understanding of 
different perspectives. 



Assignment asks students to give 
rationales for their analysis and 
interpretations, including an 
understanding of different 
perspectives. 



Extended Writing 



Assignment asks for minimal 
writing, such as multiple- 
choice or fill in the blank 



Assignment asks students 
to produce individual 
sentences. 



Assignment asks students to 
produce paragraphs 



Assignment asks students to produce 
connected paragraphs 



Disciplined Inquiry: 
Elaborated Written 
Communication 



The student work creates 
an argument, makes a 
generalization, or draws a 
conclusion and supports it 
with evidence. 



Value Beyond School 



Representations of 
Content: Depth and 
Accuracy 



Scaffolding 



The assignment requires no 
elaborated written 
communication. 



Assignment does not appear 
to be connect to adolescent 
lives 



Assignment misrepresents 
the content being studied. 



Assignment lacks necessary 
supports to help students be 
successful. 



The student is asked to 
either: A) create an 
argument, make a 
generalization, or to draw a 
conclusion or B) to provide 
evidence supporting such 
an elaboration. 



Assignment is connected to 
student interests, but does 
not support learning for 
future life 



Assignment does not cover 
all aspects of the content 
being studied. 



Assignment offers few 
supports in the way of 
graphic organizers or the 
use of multiple drafts. 



The student is asked to draw 
conclusions or to make 
generalizations or arguments. 



The skill or content of the 
assignment is connected to 
students' lives and helps prepare 
them for issues they may face 
beyond school 

Assignment covers all aspects at a 
surface level and doesn't provide 
opportunities for students to 
probe. 



Assignment offers structural 
supports, such as graphic 
organizers, multiple drafts, etc. 



The student is asked to draw 
conclusions or to make 
generalizations or arguments and to 
support them with evidence or 
illustration. 



The skill and content of the 
assignment is connected to students' 
lives and helps prepare them for 
issues they may face beyond school 

Assignment covers all relevant 
aspects of the content being studied 
and provides opportunities for 
students to probe in depth into the 
content. 

Assignment offers structural 
supports, such as graphic organizers, 
multiple drafts, etc., that support 
further investigation and revision. 



Insufficient 

Evidence 



Insufficient 

Evidence 



Insufficient 

Evidence 



Insufficient 

Evidence 



Insufficient 

Evidence 
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Table 3. Average PLATO Scores Across Teachers (out of 7 point scale) 



Element 


Mean 


Std. Dev. 


Purpose 


4.067 


(0.99) 


Intellectual Challenge 


3.671 


(1.05) 


Representations of Content 


4.002 


(0.85) 


Connections to Prior/Personal Knowledge 


3.425 


(1.25) 


Modeling 


2.597 


(0.81) 


Explicit Strategy Instruction 


2.151 


(0.80) 


Guided Practice 


2.918 


(0.96) 


Feedback 


2.984 


(1.14) 


Classroom Discourse 


2.985 


(1.15) 


ELL Accommodation 


1.679 


(1.01) 



Table 4. Average CLASS Scores Across Teachers (on a 7 point scale) 



Dimension 


Mean 


Std. Dev. 


Positive Climate 


4.417 


(1.31) 


Negative Climate 


2.109 


(1.14) 


Regard for Adolescent Perspectives 


3.449 


(1.08) 


Behavior Management 


4.591 


(1.60) 


Productivity 


4.308 


(1.34) 


Student Engagement 


4.267 


(1.26) 
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Table 5. Effect of Explicit Strategy Instruction and another PLATO Element or CLASS Dimension on the 
Odds Ratios Predicting High Value-Added Quartile 



Explicit Strategy 
Instruction 



4.88 

(0.070) 



3.31 

(0.146) 



3.47 

(0.108) 



5.91 

(0.058) 



10.17 

(0.065) 



3.16 

(0.171) 



3.76 

(0.089) 



0.76 

(0.643) 



1.24 

( 0 . 688 ) 



1.28 

(0.699) 



0.68 

(0.410) 




0.33 

(0.308) 



1.37 

(0.613) 



1.06 

(0.891) 



3.37 

( 0 . 11 ) 



3.38 

(0.115) 



3.35 

(0.117) 



1.23 

(0.523) 



1.25 

(0.577) 



1.54 

(0.337) 
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Figure 1. Distribution of Scores for each of the PLATO Elements across Value Added Quartiles 
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Figure 2. Distribution of Scores for each of the CLASS Dimensions across Value Added Quartiles 
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Figure 3. Average Score by Value Added Quartile for each of the PLATO Elements 
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Figure 4. Average Score by Value Added Quartile for each of the CLASS Dimensions 
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Figure 5. Content Domain Focus As Reported in Teacher Logs By Value Added Quartile 
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Figure 6. Use of Small and Large Groups As Reported in Teacher Logs By 
Value Added Quartile 
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